In the past, various types of spatial information detecting devices for measuring distance information of an object or extracting an outline of the object from an output of image pickup means have been proposed. For example, Japanese Patent Publication Laid-open No. 11-284997 discloses a technique of extracting the outline of the object from a gray image generated by use of an image sensor. In addition, Japanese Patent Publication Laid-open No. 64-10108 discloses a technique of determining a distance with the object by irradiating a spot-like or linear light pattern to an object, receiving a light reflected from the object by a position sensitive detector (PSD), and converting an output of the position sensitive detector into the distance in accordance with a triangular surveying method. In addition, PCT Gazette WO03/085413 discloses a spatial information detecting device for detecting spatial information such as distance from an electrical output corresponding to an intensity of received light, which is obtained by irradiating a light intensity-modulated at an emission frequency to a target space, and receiving the light reflected from an object in the target space.
By the way, a greater amount of spatial information can be obtained by using both of the gray image and the distance information. However, according to the conventional techniques, since each of gray values of the gray image and a corresponding distance value are not obtained from the same pixel, a treatment of associating each of positions in the gray image with corresponding distance value is separately needed. For example, since a light is scanned in a target space in the apparatus using the triangular surveying method, a relatively large time lag between the generation of the gray image and the generation of the distance information occurs, so that the associating treatment therebetween becomes complex. In addition, when using both of the device for generating the gray image such as a TV camera with a CCD image sensor and the device for detecting the distance information such as the position sensitive detector, an increase in size and cost of the whole apparatus also becomes a problem.